Image forming apparatus, image forming system, and communication processing method allowing to capture screen for remote maintenance

ABSTRACT

Provided is an image forming apparatus that preserves operation of a remote support. A connecting part connects with the terminal for remote maintenance in a session of a specific protocol. An operation acquiring part acquires operational information corresponding to operation of the connected terminal. A capturing part captures an emulated image. An image transmitting part transmits image data including the emulated image captured by the capturing part.

BACKGROUND

The present disclosure relates to an image forming apparatus, an imageforming system, and a communication processing method, especially isassociated with the image forming apparatus, the image forming system,and the communication processing method, which enables remotemaintenance.

An image forming apparatus, such as an MFP (Multi-FunctionalPeripheral), is used to print a document and an image.

Here, as typical technology, an image forming system that can reproducea sequence of operations performed for apparatus as controlled objects,such as an image forming apparatus, is disclosed. In this technology, acontrol part of a remote terminal transmits coordinate data of positionwhere click operation of the mouse is performed to a copying machine.The control part of the copying machine receives the coordinate datafrom the remote terminal. Then, respective coordinate data is comparedwith remote control screen data. In this comparison, whether the clickedposition is on the remote setting panel or it is on a hard-key panel isdetermined. Operational information corresponding to the decision resultis stored as script data. Then, the operational information in scriptdata is read one at a time. Processing operation based on respectiveoperational information is performed, sequentially. As a result, thesequence of operations performed by the user is reproduced. Asconfigured in this way, improvement in the operativity of the apparatuscan be promoted.

SUMMARY

An image forming apparatus in the present disclosure includes aconnecting part configured to connect with a terminal for remotemaintenance in a session of a specific protocol; an operation acquiringpart configured to acquire operational information corresponding tooperation of the terminal connected by the connecting part; a capturingpart configured to capture an emulated image emulated on an operationpanel of a self-apparatus, drawn as corresponding to the operationalinformation acquired by the operation acquiring part, and displayed on aweb browser of the terminal; and an image transmitting part configuredto transmit image data including the emulated image captured by thecapturing part.

An image forming system in the present disclosure is an image formingsystem having an image forming apparatus and a terminal for remotemaintenance of the image forming apparatus, wherein the image formingapparatus includes: a connecting part configured to connect with theterminal in a session of a specific protocol, an operation acquiringpart configured to acquire operational information corresponding tooperation of the terminal connected by the connecting part, a capturingpart configured to capture an emulated image emulated on an operationpanel of a self-apparatus, drawn as corresponding to the operationalinformation acquired by the operation acquiring part, and displayed on aweb browser of the terminal, and an image transmitting part configuredto transmit image data including the emulated image captured by thecapturing part; and the terminal includes: an operation transmittingpart configured to transmit the pointer data about movement anddepression of a pointer in the emulated image displayed on the webbrowser to the image forming apparatus in the session as the operationalinformation, and a capture transmitting part configured to transmit theemulated image to the image forming apparatus as extended data with anattribute other than the pointer data transmitted by the operationtransmitting part.

A communication processing method in the present disclosure is acommunication processing method executed by an image forming apparatuscapable of communicating to a terminal for remote maintenance,comprising the steps of: connecting with the terminal in a session of aspecific protocol; acquiring operational information corresponding tooperation of the connected terminal; capturing an emulated imageemulated an operation panel of a self-apparatus, drawn as correspondingto the acquired operational information, and displayed on a web browserof the terminal; and transmitting image data including the capturedemulated image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system configuration figure of the image forming systemaccording to an embodiment of the present disclosure;

FIG. 2 is a block diagram showing a control configuration of the imageforming apparatus as shown in FIG. 1;

FIG. 3 is a block diagram showing a control configuration of the supportterminal as shown in FIG. 1;

FIG. 4 is a block diagram showing a functional configuration of theimage forming system according to the embodiment of the presentdisclosure;

FIG. 5 shows a screen example of the emulated image shown in FIG. 4; and

FIG. 6 is a flow chart of a capture transmitting process according tothe embodiment in the present disclosure; and

FIG. 7 is a block diagram showing a functional configuration of an imageforming system according to other embodiment in the present disclosure.

SPECIFICATION Embodiment

[System Configuration of Image Forming System X]

Firstly, as refer to FIG. 1-FIG. 3, the system configuration of imageforming system X according to the embodiment of the present disclosureis explained.

According to FIG. 1, image forming system X according to the embodimentof the present disclosure includes image forming apparatus 1 and supportterminal 2 for remote maintenance of image forming apparatus 1. Also,each apparatus is connected with network 5.

Image forming apparatus 1 is information processing apparatus, such asan MFP, a network scanner, a document scanner, network FAX, a printer,or the like. Also, image forming apparatus 1 can perform copying adocument, printing, scanning, transmitting facsimile, or the like.

Also, in image forming apparatus 1 in the present embodiment, remotemaintenance between remote places via network 5 is available. In serviceof the remote maintenance, an IT administrator, or the like, who is acustomer, is in the side of image forming apparatus 1, and he or sheperform maintenance by receiving instructions from a support personnel.

Support terminal 2 is PC, a mobile phone, a smart phone, PDA (PersonalData Assistant), a business-use support terminal, a dedicated terminal,or the like. Also, in support terminal 2, a kind of general-purpose OS(Operating System), or the like, is executed. Also, support terminal 2can install a various application software (henceforth, an“application”).

Also, support terminal 2 is a terminal for remote maintenance. Thesupport personnel of the remote maintenance service as mentioned aboveuse support terminal 2 and instructs to the IT administrator, or thelike, for image forming apparatus 1. In this case, support terminal 2displays a screen just emulated operation panel part 16 (FIG. 2) inimage forming apparatus 1 and indicates various instructions on thescreen. The contents of instructions by the support personnel via thescreen are reflected on display part 47, or the like, in image formingapparatus 1. Also, image forming apparatus 1 is controlled correspondingto the button on the screen.

That is, in the present embodiment, the support may be performed withone to one communication by using function as a remote desktop functionprovided in an OS of a common PC. Therefore, for example, a RFB server,or the like is executed in image forming apparatus 1, and an RFB clientis executed in support terminal 2. Also, support terminal 2 may executethe RFB client by using Java Script®, or the like. In this case, theimage emulated operation panel part 16 can be displayed on supportterminal 2, and image forming apparatus 1 can be controlled based on thedisplayed image. That is, support terminal 2 displays the emulatedscreen as a “remote panel” by using a web browser, and it can controlimage forming apparatus 1. In detail, it is possible to use functions,such as a print, a scan, FAX transmission and reception, and networkFAX, in image forming apparatus 1 with instructions by the supportpersonnel from support terminal 2. By using these functions, it becomespossible to check and to show solution of a problem situation for acustomer's image forming apparatus 1 from a remote environment.

Network 5 is, in the present embodiment, LAN (Local Area Network), suchas intranet, or the like, or WAN (Wide Area Network), such as theInternet, a mobile phone network, or the like. In case that network 5 isWAN, it connects with other servers via a router, a gateway, or thelike. Also, network 5 may configure VPN (Virtual Private Network).

In addition, in image forming system X, it may include a plurality ofimage forming apparatuses 1 and support terminals 2, respectively.

(Configuration of Image Forming Apparatus 1)

Then, as refer to FIG. 2, a configuration of image forming apparatus 1according to the embodiment of the present disclosure is explained.Image forming apparatus 1 may include image processing part 11,manuscript reading part 12, manuscript feeding part 13, feeding part 14,network transmitting and receiving part 15, operation panel part 16,image formation part 17 (image forming unit), FAX transmission andreception part 18, memory part 19, or the like. Each part is connectedto control part 10, and these operations are controlled by control part10.

Control part 10 is an information processing part, such as GPP (GeneralPurpose Processor), CPU (Central Processing Unit), MPU (Micro ProcessingUnit), DSP (Digital Signal Processor), GPU (Graphics Processing Unit),ASIC (Application Specific Processor), or the like. Control part 10reads the control program stored in ROM or HDD of memory part 19,expands to RAM, executes the control program, and is operated as eachpart of a functional block as described later. Also, control part 10controls a whole device corresponding to specified instructionsinformation inputted from support terminal 2 or operation panel part 16.

Image processing part 11 is a control arithmetic unit, such as DSP(Digital Signal Processor) or GPU (Graphics Processing Unit), or thelike. Image processing part 11 performs specified image processing forimage data. For example, the specified image processing may be a processof scaling, concentration tuning, tone adjustment, an image improvement,or the like. Also, image processing part 11 stores the image read bymanuscript reading part 12 as print data in memory part 19. In thiscase, image processing part 11 can also convert print data into anelectronic filing documents, such as PDF, or the like, or a file of animage data, such as TIFF, or the like. Also, image processing part 11may be able to execute at least a part of a process of OCR (OpticalCharacter Recognition).

Manuscript reading part 12 reads a set manuscript. Manuscript readingpart 12 is arranged on the upper part of the body part of image formingapparatus 1. Manuscript reading part 12 is provided with a scanner,platen glass, and a manuscript reading slit. Manuscript reading part 12moves the scanner to the position that faces the platen glass whenreading the manuscript placed on the platen glass. Manuscript readingpart 12 is read the manuscript placed on the platen glass with scanningand acquires image data. Manuscript reading part 12 outputs the acquiredimage data to control part 10.

Also, manuscript reading part 12 moves the scanner to the position whichfaces a manuscript reading slit when reading the manuscript fed frommanuscript feeding part 13. Then, via the manuscript reading slit,manuscript reading part 12 reads the manuscript synchronizing withconveyance operation of the manuscript by manuscript feeding part 13,and it acquires image data. Manuscript reading part 12 outputs theacquired image data to control part 10.

Manuscript feeding part 13 conveys the manuscript read by manuscriptreading part 12. Manuscript feeding part 13 is arranged on the upperpart of manuscript reading part 12. Manuscript feeding part 13 isprovided with a manuscript mounting part and a manuscript transportmechanism. Manuscript feeding part 13 feeds out the manuscript placed onthe manuscript mounting part one sheet at a time by using the manuscripttransport mechanism, and it feeds to manuscript reading part 12.

Feeding part 14 feeds out a recording paper one sheet at a time to imageformation part 17. Feeding part 14 is provided in the body part.

Network transmitting and receiving part 15 is a network connection partincluding a LAN board, a wireless transceiver, or the like, forconnecting with external networks, such as LAN, wireless LAN, WAN, themobile phone network, or the like. Network transmitting and receivingpart 15 transmits and receives data by using a line for datacommunications, and transmits and receives an audio signal by using avoice call line.

Operation panel part 16 is arranged in front-side of image formingapparatus 1. Also, operation panel part 16 includes input part 46 anddisplay part 47.

Input part 46 is an input unit of a touch panel, physical buttons, orthe like, which acquires the instructions by IT administrator, or thelike, to image forming apparatus 1. Input part 46 may include buttons,such as a start, a stop, a reset, a copy, a transmission, and a FAX, orthe like, to instruct for calling various functions. Also, input part 46may include a physical numeric keypad, a “#Keys” button for displaying anumeric keypad on display part 47, a “home” button for returning to the“home” screen that is the first screen, or the like. Also, input part 46may include various buttons, such as an “Interrupt” of a job, an“Authentication/Logout,” an “Energy saver,” or the like. Also, inputpart 46 may include buttons for making a various state of performing aninstruction about the job change, such as an “Accessibility Display,” a“Status/Job cancel,” a “System Menu/Counter,” or the like. Also, the jobto be instructed may be types, such as printing, transmitting, storing,and recording for a selected document.

In addition, it is also possible to input and change each user'sinformation with a user's instructions acquired from input part 46.Also, input part 46 can connect a reader for a magnetic card or an ICcard, a device for biometric authentication, or the like.

Display part 47 includes a display panel, such as LCD (Liquid CrystalDisplay), an organic EL display, and an indicator, such as LED (LightEmitting Diode), or the like. The display panel of display part 47 candisplay a various operation screens of GUI (Graphical User Interface).In the present embodiment, input part 46 and display part 47 may becombined and configured as like a display having a touch panel.Therefore, it is also possible to arrange buttons for various operationson display part 47 and is instructed via input part 46 by depressingthem. Also, as above-mentioned, it is possible to display a numerickeypad on display part 47. Also, LED of display part 47 can displayvarious states. Also, LED can display the various states of theself-apparatus. Also, some of LED, or the like, for display part 47 maybe provided inside of the button. Thereby, it can inspect a state ofdepression of the button, or the like.

Also, operation panel part 16 may be provided with a connecting partthat connects with an external record medium, such as flash memory card,a USB storage apparatus, or the like.

Image formation part 17 makes the image formation from the data storedin memory part 19, read by manuscript reading part 12, or acquired fromthe external support terminal by a user's output instruction to arecording paper perform. Image formation part 17 is provided with aphoto conductor drum, an exposure part, a developing part, a transferpart, a fixing part, or the like. Image formation part 17 records atoner image on a recording paper by performing an image formationprocess of electrification, exposure, development, transfer, and fixing.

FAX transmission and reception part 18 transmits and receives afacsimile. FAX transmission and reception part 18 can perform facsimilereceiving from other FAX equipment via a voice line, can preserve it onmemory part 19, and can perform image formation by image formation part17. Also, FAX transmission and reception part 18 converts the manuscriptread by manuscript reading part 12, the data of network FAX transmittedfrom the external support terminal into image data. FAX transmission andreception part 18 can perform facsimile transmission of the image datato other FAX equipment via a voice line.

Memory part 19 is a memory part having a non-transitory recordingmedium. Memory part 19 may include, for example, semiconductor memory,such as ROM (Read Only Memory), RAM (Random Access Memory), or the like,or HDD (Hard Disk Drive), or the like. RAM of memory part 19 may be helda memory content in a power-saving state by self refreshing function, orthe like. The control program for operation-controlling image formingapparatus 1 is stored in ROM or HDD of memory part 19. The controlprogram may be an OS and various applications. In addition, memory part19 may also store user account settings. Also, an area of a storagefolder (Document box) for each user may be included in memory part 19.

In addition, image forming apparatus 1, control part 10 and imageprocessing part 11 may be integrally-formed as like CPU with built-inGPU, a chip-on module package, SOC (System On Chip), or the like. Also,control part 10 and image processing part 11 may contain RAM, ROM, aflash memory, or the like.

(Configuration of Support Terminal 2)

Also, according to FIG. 3, support terminal 2 is provided with controlpart 20, network transmitting and receiving part 25, input part 26,display part 27, memory part 29, or the like.

Control part 20 may be an information processing part, such as GPP(General Purpose Processor), CPU (Central Processing Unit), MPU (MicroProcessing Unit), DSP (Digital Signal Processor), GPU (GraphicsProcessing Unit), ASIC (Application Specific Processor, processor forparticular applications), or the like.

Input part 26 is a keyboard, various sensors, a pointing device, or thelike, for acquiring the various instructions by a user. The pointingdevice includes a touch panel, a digitizer, a touchpad, or the like.Input part 26 can make various instructions input by a user, such as thesupport personnel, or the like, by using GUI (Graphical User Interface)of OS and can acquire this. Also, information of moving the pointer anddepressing a key (physical key) of the keyboard in input part 26 istransmitted to image forming apparatus 1. That is, the display ofdisplay part 27 explained below is updated based on this information.

Display part 27 includes a flat-panel screen panel, such as LCD, anorganic EL display, a FED (Field Emission Display), a fluorescentcharacter display tube, or the like, a projector, LED for a statusdisplay, or the like. Display part 27 can display various operationscreens according with GUI. In addition, input part 26 and display part27 may be integrally-formed as like a display with the touch panel orthe digitizer.

Network transmitting and receiving part 25 is a network connection partincluding a LAN board, a wireless transceiver, or the like, forconnecting with network 5.

Memory part 29 is a memory part having a non-transitory recordingmedium. Memory part 29, for example, may include various RAM as a mainmemory part. Also, memory part 29, for example, may include ROM, eMMC(embedded Multi Media Card), SSD (Solid State Drive), HDD, or the likeas an auxiliary memory part. Also, memory part 29 stores a controlprogram for the self-apparatus. The control program may be various OS, aweb browser that can browse webs, such as WWW, or the like, variousapplications, or the like. Also, memory part 29 may include externalstorages, such as various flash memories, optical recording media, orthe like.

[Functional Configuration of Image Forming System X]

Control part 10 of image forming apparatus 1 is provided with connectingpart 100, operation acquiring part 110, capturing part 180, and imagetransmitting part 190. Memory part 19 of image forming apparatus 1stores image data 580. Control part 20 of support terminal 2 is providedwith display operating part 200, operation transmitting part 210, andcapture transmitting part 280. Memory part 29 of support terminal 2stores operational information 400 and emulated image 410.

Connecting part 100 connects with support terminal 2 in a session of aspecific protocol. The specific protocol may be a protocol extended fromthe RFB protocol. In this case, connecting part 100 may execute aprogram compatible with the RFB server. Also, connecting part 100 cantransmit emulated image 410 to support terminal 2 according to theinstructions from operation acquiring part 110. The transmission may beexecuted as corresponded to the RFB protocol.

Operation acquiring part 110 acquires operational information 400corresponding to operation of support terminal 2 connected by connectingpart 100. Also, operation acquiring part 110 interprets operationalinformation 400, updates display part 47 of operation panel part 16, andgenerates and updates emulated image 410 corresponding to this. In thiscase, operation acquiring part 110 may generate and update emulatedimage 410 based on states of a display panel, LED, or the like, indisplay part 47.

Capturing part 180 captures emulated image 410. Here, in the presentembodiment, emulated image 410 captured by capturing part 180 is animage drawn as corresponding to operational information 400 acquired byoperation acquiring part 110, as described later. Also, in the presentembodiment, emulated image 410 is the image emulated on operation panelpart 16 of the self-apparatus and displayed on the web browser insupport terminal 2. In detail, capturing part 180 may acquire emulatedimage 410 transmitted from capture transmitting part 280 in supportterminal 2 and may include it in image data 580 in memory part 19. Inthis case, capturing part 180 may capture emulated image 410 withextended data other than the pointer data included in operationalinformation 400. Also, capturing part 180 may accumulate emulated image410 in image data 580 and may convert it into video data. The video datamay be stored in various containers, such as MJPEG or MPG, or the like.Also, capturing part 180 may include voice data 470 in video data asdescribed later.

Image transmitting part 190 transmits image data 580 including emulatedimage 410 captured by capturing part 180. In detail, image transmittingpart 190 may transmit image data 580 with various protocols. In case ofthe protocol of an e-mail (SMTP), messenger, SMS (Short MessageService), or the like (Hereinafter, they are just called as “e-mail”),image transmitting part 190 may transmit to an e-mail addressing to ane-mail address of IT administrator, or the like. Also, imagetransmitting part 190 may transmit image data 580 to NAS, a server, orthe like, with the protocol of FTP (File Transfer Protocol), SMB, or thelike (Hereinafter, they are just called as “FTP”). Also, imagetransmitting part 190 may transmit image data 580 to a cloud storage ina server, a web service server, or the like (Hereinafter, they are justcalled as “cloud”) with the various protocols. Also, image transmittingpart 190 may enable the support personnel, IT administrator, or thelike, to refer to account settings or an address book stored in thedocument box in memory part 19, or an address book in another serveraccessible from image forming apparatus 1, or the like, and to select anaddress for image data 580.

Display operating part 200 connects with connecting part 100 in imageforming apparatus 1. In the present embodiment, display operating part200 may connect with connecting part 100 with the protocol extended fromthe RFB protocol. In this case, display operating part 200 may executethe “remote panel,” which is a program compatible with the RFB clientfor image forming apparatus 1. Also, display operating part 200 acquiresemulated image 410 from above-mentioned connecting part 100 in imageforming apparatus 1 and displays it on display part 27. Also, displayoperating part 200 may execute the remote panel on the web browser insupport terminal 2 by using Java Script®, or the like. The programcompatible with the RFB client is acquired by connecting with the webserver in image forming apparatus 1 or, otherwise, it may be acquired byaccessing a web site of a server for a support. Thereby, withoutprior-installation the dedicated application, and it becomes possible touse the remote panel.

Also, in the present embodiment, display operating part 200 can move thepointer in all areas of emulated image 410 displayed on the web browserin support terminal 2. The areas may include areas other than the areafor image of the display panel, which is actually-displayed on displaypart 47 in operation panel part 16 in image forming apparatus 1. In thepresent embodiment, as described later, an image of the mode changebutton is included in an area other than the display panel.

Also, display operating part 200 may enables inputting a character,drawing an annotation picture, or drawing a highlight as a “drawingmode” in the remote panel on the web browser by using a key, a pointer,touch operation, or the like. In this case, the inputted character, thedrawn annotation picture or the highlight may be reflected in respectiveemulated image 410 by display operating part 200. Also, displayoperating part 200 may have a mode that the operation guidance on aremote panel itself does not reflect on image forming apparatus 1 butonly the operating procedure is shown as a “guidance mode.”

Also, in the present embodiment, conversation with voice sound betweenthe support personnel and IT administrator may be performed betweendisplay operating part 200 in support terminal 2 and connecting part 100in image forming apparatus 1. Therefore, connecting part 100 and displayoperating part 200 transmit and receive a packet of voice data 470between support terminal 2 and image forming apparatus 1 with theprotocol extended on the RFB protocol in the session under connection.

Operation transmitting part 210 acquires various instructions of thesupport personnel on the remote panel and transmits them to imageforming apparatus 1. In detail, in the present embodiment, operationtransmitting part 210 acquires pointer data about movement anddepression of the pointer in emulated image 410 rendered on display part27. Further, operation transmitting part 210 transmits the pointer dataas operational information 400 to image forming apparatus 1 in thesession. Also, operation transmitting part 210 can acquire the pointerdata having for an area of the physical buttons of operation panel part16 on emulated image 410.

Also, operation transmitting part 210 can also acquire pointer data onthe area for the image of the mode change button in emulated image 410as operational information 400.

Capture transmitting part 280 transmits emulated image 410 to imageforming apparatus 1 as extended data. The extended data may have anattribute other than the pointer data of operational information 400transmitted by operation transmitting part 210. In detail, capturetransmitting part 280 transmits, for example, a whole emulated image 410to image forming apparatus 1 by the extended message type of theprotocol extended on the RFB protocol. In this case, emulated image 410transmitted by capture transmitting part 280 may be compressed as jpg,PNG, or the like, or may be video data, such as MJPEG, MPG, or the like.Also, emulated image 410 that capture transmitting part 280 transmitsmay be differential data of data transmitted at last time.

Also, in the present embodiment, display part 27 displays emulated image410.

Operational information 400 is information for an operation by thesupport personnel in the remote panel. In detail, operationalinformation 400 may include the pointer data about movement anddepression of the pointer in emulated image 410, or the like. Also,operational information 400 may include the information of depression ofa button as the pointer data. As information of depression of thebutton, the information of depression for a physical key in input part26 may be included. The information of depression for the physical keymay be acquired as like the information of depression of the buttondrawn on emulated image 410 as corresponded to a short-cut key, or thelike.

Also, as above-mentioned, operational information 400 may include thepointer data on the area for the image for the mode change button, whichchanges the mode, or the like.

Emulated image 410 is an image data of emulated operation panel part 16for image forming apparatus 1 displayed on the web browser. Emulatedimage 410 may also include an image that emulates an appearance of inputpart 46 and display part 47 of operation panel part 16. Also, emulatedimage 410 is a dot matrix image corresponding to the RFB protocol andmay be partially- or entirely-updateable on image forming apparatus 1and support terminal 2. The detailed example of emulated image 410 isdescribed later.

Image data 580 is data including emulated image 410. The emulated image410 may be an image of the remote panel for remote maintenance with theinstructions by the support personnel in support terminal 2. That is,emulated image 410 included in image data 580 may be an image reflectingthe character inputting, drawing, or the like, in a “drawing mode” onsupport terminal 2. Also, emulated image 410 may be an image at the timeof being operated to show various operating procedures in a “guidancemode.”

Also, image data 580 may be video data at the time of the remotemaintenance. Also, the video data may include voice data 470 transmittedand received between support terminal 2 and the self-apparatus.

Also, emulated image 410 includes an image of capture button C forcapturing an image in an area other than displaying by operation panelpart 16. The area of the capture button C may be on a place other thanthe areas of panel D, indicator L, and button group B. Also, the capturebutton C is included only in emulated image 410, and it does not need tobe present in operation panel part 16 in image forming apparatus 1. Inaddition, capture button C may be arranged other than the inside ofemulated image 410, or it may be arranged on a menu, a button, or thelike, in the web browser. Also, a configuration that depression of thekey, or the like, in input part 26 makes an equivalent instruction bydepression of capture button C included in operational information 400is possible.

Here, control part 10 in image forming apparatus 1 executes a controlprogram stored in memory part 19 and functions as connecting part 100,operation acquiring part 110, capturing part 180, and image transmittingpart 190. Also, control part 20 in support terminal 2 executes a controlprogram stored in memory part 29 and functions as display operating part200, operation transmitting part 210, and capture transmitting part 280.

Also, each part of the above-mentioned image forming apparatus 1 andsupport terminal 2 serves as hardware resources to execute thecommunication processing method in the present disclosure. In addition,the above-mentioned part or arbitrary combination of a functionalconfiguration may be configured as in hardware with an IC, aprogrammable logic, FPGA (Field Programmable Gate Array), or the like.

[Capture Transmitting Process in Image Forming System X]

Then, as refer to FIG. 6, a capture transmitting process by imageforming system X according to the embodiment of the present disclosureis explained. The capture transmitting process of the presentembodiment, mainly, control part 10 of image forming apparatus 1executes the control program stored in memory part 19, control part 20of support terminal 2 executes the control program stored in memory part29 with collaborating each part and using hardware resources. In thefollowing, as refer to a flow chart in FIG. 6, the details of thecapture transmitting process are explained for each step.

(Step S151)

Firstly, connecting part 100 of image forming apparatus 1 performs aconnecting process. The IT administrator, or the like, uses service ofremote maintenance in time of problem solving or training of imageforming apparatus 1, or the like. In this case, IT administrator, or thelike, may also possibly receive and answer by voice message, directly,with the support personnel in the support terminal 2 by using a voiceline, or the like, connected with the image forming apparatus 1. Thesupport personnel may communicate that starting remote maintenance byusing the remote panel function of image forming apparatus 1 is desired.Then, IT administrator, or the like, instructs to execute the remotepanel as remote maintenance by using operation panel part 16 in imageforming apparatus 1, or the like. Thereby, connecting part 100 connectswith support terminal 2 in a session of the specific protocol, such as aprotocol extended from the RFB protocol. In this case, connecting part100 executes a program compatible with the RFB server.

Also, in this case, connecting part 100 in image forming apparatus 1 anddisplay operating part 200 in support terminal 2 may connect to transmitand receive voice data 470. The connection may be the same asabove-mentioned session. In addition, a voice call by a messenger, orthe like, may be performed, separately. Also in this case, connectingpart 100 and display operating part 200 may acquire voice data 470 bysettings of the mixer of OS, or the like, and may perform transmissionand reception between image forming apparatus 1 and support terminal 2.

(Step S251)

Here, display operating part 200 in support terminal 2 performs adisplay operation acquiring process. The support personnel of supportterminal 2 starts the web browser and executes the “remote panel” thatis a program compatible with the RFB client. Thereby, display operatingpart 200 connects with connecting part 100 in image forming apparatus 1with the protocol extended from the RFB protocol. Thereby, emulatedimage 410 is displayed on the web browser, and movement, depression ofthe pointer, or the like, is enabled by input part 26.

(Step S252)

Then, operation transmitting part 210 performs an operationalinformation transmitting process. Operation transmitting part 210acquires rectangle coordinates of the pointer moved in emulated image410 on the web browser as the pointer data. Also, operation transmittingpart 210 acquires information of depression of a button or a physicalkey and adds it to the pointer data, too. Operation transmitting part210 transmits the pointer data as operational information 400 to imageforming apparatus 1.

Here, operation transmitting part 210 may also acquire pointer data inthe area of the image for the capture button in emulated image 410 asoperational information 400, and operation transmitting part 210 maytransmit it.

(Step S152)

Then, capturing part 180 determines whether the capture is started ornot. In operational information 400, capturing part 180 determines Yesif the area of the image for the capture button is depressed with thepointer. Capturing part 180 determines No if other. In Yes, capturingpart 180 advances a process to Step S153. In No, capturing part 180 endsthe capture transmitting process.

(Step S153)

In case that capturing is instructed, capturing part 180 performscapture starting process. Capturing part 180 starts waiting toaccumulate emulated image 410 from support terminal 2 in image data 580.In this case, capturing part 180 also starts the waiting to accumulatevoice data 470 in image data 580. Then, as explained later, capturingpart 180 acquires emulated image 410 transmitted from support terminal 2and accumulates it in image data 580. In this case, capturing part 180may be stored in various containers after converting or compressingemulated image 410 to video data.

(Step S253)

Here, capture transmitting part 280 in support terminal 2 performs anemulated image transmitting process. Capture transmitting part 280transmits emulated image 410 to image forming apparatus 1. Emulatedimage 410 in this case may not be emulated image 410 itself acquiredfrom image forming apparatus 1 but may be an image displayed on displaypart 27 as the remote panel on the web browser. Also, emulated image 410in this case may be a drawn image, which the user draws to the remotepanel on the web browser. That is, capture transmitting part 280 maytransmit emulated image 410 processed in support terminal 2. Also, inthe case of transmission of emulated image 410, capture transmittingpart 280 may compress or convert emulated image 410 into video data, orextract only difference data, and then transmit it.

(Step S154)

Here, capturing part 180 of image forming apparatus 1 performs a voicedata acquiring process. Capturing part 180 acquires voice data 470transmitted and received between image forming apparatus 1 and supportterminal 2, and includes it in image data 580. In this case, capturingpart 180 may set up voice data 470 to be synchronized on the time lineof the video data.

(Step S155)

Then, image transmitting part 190 performs an image data transmittingprocess. Image transmitting part 190 transmits image data 580, forexample, with reference to account settings. In this case, imagetransmitting part 190 attaches image data 580 and transmits it to e-mailaddresses, such as IT administrator, or the like, by an e-mail. Also,image transmitting part 190 may transmit image data 580 by FTP, SMB, orthe like. Also, image transmitting part 190 may transmit image data 580to the cloud storage. In addition, image data 580 accumulated on thecloud storage is distributed to other customers having the same problemif consent of the customer, such as IT administrator, or the like, isobtained. Therefore, on the cloud storage, for each case of remotemaintenance, the support personnel may tag image data 580 as the“snapshot” and may put in a database and store it. As above-mentioned,the capture transmitting process according to the embodiment in thepresent disclosure is ended.

As configured as mentioned above, the following effects can be obtained.

In typical technology, the screen of remote maintenance is not able tobe captured.

On the other hand, image forming system X according to the embodiment inthe present disclosure includes image forming apparatus 1 and supportterminal 2 for the remote maintenance for image forming apparatus 1.Image forming apparatus 1 is provided with connecting part 100,operation acquiring part 110, capturing part 180, and image transmittingpart 190. Connecting part 100 connects with support terminal 2 forremote maintenance in a session of a specific protocol. Operationacquiring part 110 acquires operational information 400 corresponding tooperation of support terminal 2 connected by connecting part 100.Capturing part 180 captures emulated image 410. The emulated image 410is drawn as corresponded to operational information 400 acquired byoperation acquiring part 110 and is emulated operation panel part 16 ofthe self-apparatus displayed on the web browser in support terminal 2.Image transmitting part 190 transmits image data 580 including emulatedimage 410 captured by capturing part 180. Support terminal 2 is providedwith operation transmitting part 210 and capture transmitting part 280.Operation transmitting part 210 transmits pointer data as operationalinformation 400 to image forming apparatus 1 in the session. The pointerdata is data about movement and depression of the pointer in emulatedimage 410 displayed on the web browser. Capture transmitting part 280transmits emulated image 410 to image forming apparatus 1 as extendeddata with attributes other than the pointer data transmitted byoperation transmitting part 210.

As configured in this way, unlike accumulating a script or the like,preserving image data 580 in accordance with practical actions for afile in image forming apparatus 1, or the like, can be performed.

That is, the image forming apparatus that can reproduce the screen ofremote maintenance, certainly, is provided. Therefore, the screen dataused for the support personnel for teaching operation to ITadministrator, or the like, can be transmitted to IT administrator, orthe like, as it is.

Also, in typical technology, knowledge taught operation of image formingapparatus 1 in remote maintenance from the support personnel to ITadministrator, or the like, may be lost. This cause is a transfer of ITadministrator, a person in charge, or the like, or just progress oftime, or the like. Therefore, the support personnel need to teachoperator guidance again to IT administrator, or the like. On the otherhand, education of the operator guidance for the second time, easily, ispossible by transmitting image data 580. Also, if consent of customers,such as IT administrator, or the like, is obtained, using image data 580to other customers' education or providing it to the other customer inan equivalent case for remote maintenance is possible. Thereby, theefficiency of remote maintenance can be increased.

Also, in image forming apparatus 1 according to the embodiment in thepresent disclosure, image data 580 may be video data. Also, the videodata may include voice data 470 transmitted and received between supportterminal 2 and the self-apparatus. As configured in this way, educationby the support personnel in remote maintenance, or the like, asmentioned above, can be more intelligible, and the efficiency can beincreased.

Also, in image forming apparatus 1 according to the embodiment in thepresent disclosure, image transmitting part 190 transmits image data 580by an e-mail or FTP, or to cloud storage. As configured in this way,transmitting image data 580 to IT administrator, or the like, can beperformed, or to preserve and accumulate it in the other server can beperformed. As a result, the efficiency of the education by the supportpersonnel, or the like, can be increased.

Also, in image forming apparatus 1 according to the embodiment in thepresent disclosure, the specific protocol is a protocol extended on theRFB protocol. Also, operational information 400 includes pointer dataabout movement and depression of a pointer on emulated image 410.Capturing part 180 captures emulated image 410 with extended data otherthan the pointer data. As configured in this way, the specification ofthe RFB protocol is maintained and is extended. Thereby, in the sessionbetween image forming apparatus 1 and support terminal 2, sending thecapture image can be performed. As a result, problems such asconnection, or the like, are less-likely to occur.

Other Embodiments

In addition, in the above-mentioned embodiment, a configuration thatimage forming apparatus 1 has image transmitting part 190 and transmitsimage data 580 from image forming apparatus 1 is explained. However,image data 580 may be transmitted not only from image forming apparatus1 but from support terminal 2. As refer to FIG. 7, support terminal 2 ofimage forming system Y according to other embodiments in the presentdisclosure is provided with capture transmission and reception part 281and image transmitting part 290. Further, FIG. 7 gives the same numeralsabout the component having equivalent configuration as above-mentionedother figures.

Capture transmission and reception part 281 has the function of capturetransmitting part 280 in image forming apparatus 1, also it acquiresimage data from image forming apparatus 1 as extended data, and itstores them in memory part 29.

Image transmitting part 290 transmits image data 580 as like imagetransmitting part 190 in image forming apparatus 1. In this case, imagetransmitting part 290 may transmit image data 580 by using variousprotocols, or the like, which support terminal 2 supports in addition totransmit by an e-mail or FTP, or to cloud storage, or the like. Also,image transmitting part 290 may couple and transmit it with an installedapplication, or the like. The application, or the like, for example, mayinclude a device driver, such as various kinds of cloud storage, anetwork drive, and storage of iSCSI, or the like. Also, the applicationmay be an application specifying for a support where the supportpersonnel use.

As configured in this way, to transmit image data 580 by using theprotocol, or the like, and function, which support terminal 2 supportsin addition to the protocol, or the like, which image forming apparatus1 supports is possible. Thereby, convenience of the support personnel,IT administrator, or the like, can be improved. Also, since an address,or the like, is easily selected in support terminal 2 by using GUI, orthe like, the transmission becomes easier.

Also, in image forming system Y, the support personnel or ITadministrator, or the like, may select whether image data 580 istransmitted from image forming apparatus 1 or from support terminal 2.Also, image data 580 transmitted from image forming apparatus 1 or fromsupport terminal 2 may be different. For example, when transmitting fromimage forming apparatus 1, image data 580, which does not include theimage of the area for a physical button, or the like, may betransmitted. On the contrary, when transmitting from support terminal 2,image data 580, which is a whole emulated image 410 including a physicalbutton, or the like, may be transmitted. Also, a configuration thatimage data 580 is not generated in image forming apparatus 1 but isgenerated in support terminal 2 is also possible. In this case, voicedata 470 may be acquired in support terminal 2, and it may be includedto image data 590. Also, for generating image data 580, conversion of aformat, or the like, may be performed in other external servers. Also,it may be a configuration that transmits image data 580 only fromsupport terminal 2. As configured in this way, transmitting image data580 according to user environment can be performed. Also, by generatingimage data 580 in support terminal 2, the loading of image formingapparatus 1 can be decreased.

In the above-mentioned embodiment, an example that executes the remotepanel with the RFB protocol is explained. However, a remote panel may beperformed with other original protocols, and the above-mentioned remotemaintenance may be performed. Also, in an above-mentioned embodiment, anexample that a program corresponding to an RFB client is executed on aweb browser in support terminal 2 to perform the remote panel isexplained. However, in support terminal 2, a dedicated application maybe installed, and a remote panel may be performed. For example, emulatedimage 410 may be generated in image forming apparatus 1, without usingVNC, or the like, the image may be pushed to the web browser or thededicated application, serially, or streaming may be performed insupport terminal 2. Also, contrary, in support terminal 2, an operationdisplay program that generates emulated image 410 of image formingapparatus 1 corresponding to operation of the support personnel may beexecuted. In this case, by executing the operation display program,operational information 400 may be transmitted as well as theabove-mentioned configuration, and image forming apparatus 1 may becontrolled. Also, in this case, without transmitting the image dataitself from image forming apparatus 1, only data required to generateemulated image 410, which is data related to a user file, or the like,may be transmitted. In addition, the operation display program may be a“real” emulator that executes the control program, which is similarprogram stored in memory part 19 in image forming apparatus 1, atsupport terminal 2. As configured in this way, a remote panel can beestablished even if an environment that the RFB protocol is unusable,for example, environment connectable only with the port of HTTP, or thelike. Also, it is also possible to configure so as to compress thebandwidth of data related to transmission and reception of data.

Also, in the above-mentioned embodiment, it is mainly-disclosed anexample that operation by the support personnel is reflected indisplaying or controlling to image forming apparatus 1. However, theoperation by IT administrator, or the like, to operation panel part 16in image forming apparatus 1 may be reflected in controlling of emulatedimage 410 or image forming apparatus 1. Also, it is possible toconfigure that the reflected result may be transmitted to supportterminal 2. Thereby, in addition to instruction and operation by thesupport personnel, corresponding to actual operation by ITadministrator, emulated image 410 can be generated, or image formingapparatus 1 can be controlled. This can help IT administrator intraining, or the like, understand well.

Also, as above-mentioned embodiment, a configuration that capturesemulated image 410 transmitted from support terminal 2 is disclosed.However, a configuration that captures emulated image 410 stored inmemory part 19 in image forming apparatus 1 and preserves as image data580 is also possible. Even in this case, input of a character, drawingof an annotation picture, or drawing of a highlight may be possible inthe “drawing mode” for emulated image 410 in memory part 19 by thesupport personnel's pointer data. Also, each part of image formingapparatus 1 may not be controlled in the “guidance mode.”

As configured in this way, the increase in the data traffic in thesession can be suppressed by transmitting emulated image 410 fromsupport terminal 2.

Also, in image data 580, emulated image 410 may be stored as a stillpicture as like a “flip book,” and voice data 470 may not be included.Also, image data 580 may be compressed at the time of transmission byimage transmitting part 190 or may be compressed on cloud storage.

Also, the present disclosure is applicable to information processingapparatus other than the image forming apparatus. That is, it may beconfigured to use the network scanner, a server, or the like, whichseparately-connected with a scanner by USB, or the like.

Also, the configuration and operation of the above-mentioned embodimentare an example, and it cannot be overemphasized that it can changesuitably and can execute in the range that does not deviate from the aimof the present disclosure.

What is claimed is:
 1. An image forming apparatus comprising: aconnecting part configured to connect with a terminal in a session of aspecific protocol for remote maintenance of the image forming apparatusfrom the terminal; an operation acquiring part configured to acquire, inthe session of the specific protocol for the remote maintenance,operational information corresponding to operation of the terminalconnected by the connecting part; a capturing part configured to capturean emulated image emulated based on an operation panel of aself-apparatus, drawn as corresponding to the operational informationacquired by the operation acquiring part, and displayed on a web browserof the terminal; and an image transmitting part configured to transmitimage data including the emulated image captured by the capturing part.2. The image forming apparatus according to claim 1, wherein the imagedata is video data, and the video data allows inclusion of voice datatransmitted and received between the terminal and the self-apparatus viaa voice call between the image forming apparatus and the terminal. 3.The image forming apparatus according to claim 1, wherein the imagetransmitting part transmits the image data by an e-mail or FTP, or tocloud storage.
 4. The image forming apparatus according to claim 1,wherein the specific protocol is a protocol extended from an RFBprotocol, and the operational information includes pointer data aboutmovement and depression of a pointer on the emulated image; and thecapturing part captures the emulated image with extended data other thanthe pointer data.
 5. The image forming apparatus according to claim 1,wherein the session is a remote panel session for operation of the imageforming apparatus from the terminal.
 6. An image forming system havingan image forming apparatus and a terminal for remote maintenance of theimage forming apparatus, wherein the image forming apparatus includes: aconnecting part configured to connect with a terminal in a session of aspecific protocol for remote maintenance of the image forming apparatusfrom the terminal, an operation acquiring part configured to acquire, inthe session of the specific protocol for remote maintenance, operationalinformation corresponding to operation of the terminal connected by theconnecting part, a capturing part configured to capture an emulatedimage emulated based on an operation panel of a self-apparatus, drawn ascorresponding to the operational information acquired by the operationacquiring part, and displayed on a web browser of the terminal, and animage transmitting part configured to transmit image data including theemulated image captured by the capturing part; and the terminalincludes: an operation transmitting part configured to transmit pointerdata about movement and depression of a pointer in the emulated imagedisplayed on the web browser to the image forming apparatus in thesession as the operational information, and a capture transmitting partconfigured to transmit the emulated image to the image forming apparatusas extended data with an attribute other than the pointer datatransmitted by the operation transmitting part.
 7. The image formingsystem according to claim 6, wherein the image data is video data, andthe video data allows inclusion of voice data transmitted and receivedbetween the terminal and the self-apparatus via a voice call between theimage forming apparatus and the terminal.
 8. The image forming systemaccording to claim 6, wherein the image transmitting part transmits theimage data by an e-mail or FTP, or to cloud storage.
 9. The imageforming system according to claim 6, wherein the specific protocol is aprotocol extended on an RFB protocol.
 10. The image forming systemaccording to claim 6, wherein the session is a remote panel session foroperation of the image forming apparatus from the terminal.
 11. Acommunication processing method executed by an image forming apparatuscapable of communicating to a terminal for remote maintenance of theimage forming apparatus from the terminal, comprising the steps of:connecting with the terminal in a session of a specific protocol for theremote maintenance; acquiring operational information corresponding tooperation of the connected terminal in the session of the specificprotocol for the remote maintenance; capturing an emulated imageemulating an operation panel of a self-apparatus, drawn as correspondingto the acquired operational information, and displayed on a web browserof the terminal; and transmitting image data including the capturedemulated image.
 12. The communication processing method according toclaim 11, wherein the image data is video data, and the video dataallows inclusion of voice data transmitted and received between theterminal and the self-apparatus via a voice call between the imageforming apparatus and the terminal.
 13. The communication processingmethod according to claim 11, wherein the step of transmitting comprisestransmitting the image data by an e-mail or FTP, or to cloud storage.14. The communication processing method according to claim 11, whereinthe specific protocol is a protocol extended from an RFB protocol, theoperational information includes pointer data about movement anddepression of a pointer on the emulated image; and the method comprisescapturing the emulated image with extended data other than the pointerdata.
 15. The communication processing method according to claim 11,wherein the session is a remote panel session for operation of the imageforming apparatus from the terminal.